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DETAILED ACTION 



1 . Claims 1 - 19 are pending in this office action. 



Claim Rejections - 35 USC § 101 



2. 35 U.S.C. 101 reads as follows: 

Whoever invents or discovers any new and useful process, machine, manufacture, or composition of 
matter, or any new and useful improvement thereof, may obtain a patent therefore, subject to the 
conditions and requirements of this title: 

3. Claim 10 is rejected under 35 U.S.C. 101 because the claimed invention is 
directed to non-statutory subject matter. 

A process that merely manipulates an abstract idea or performs a purely 
mathematical algorithm is non-statutory, despite the fact that it might inherently have 
some usefulness, Sarkar, 588 F.2d at 1335, 200 USPQ at 139. For such subject matter 
to be statutory, the claimed must be limited to a practical application of the abstract idea 
or mathematical algorithm in the technological arts, Alappat, 33 F.3d at 1543, 31 
USPQ2d at 1556-57. 

If the "acts" of a claimed process manipulate only numbers, abstract concepts or 
ideas, or signals representing any of the foregoing, the acts are not being applied to 
appropriate subject matter. Thus, a process consisting solely of mathematical 
operations, i.e., converting one set of numbers into another set of numbers, does not 
manipulate appropriate subject matter and thus cannot constitute a statutory process. 
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In practical terms, claims define non-statutory processes if they simply 
manipulate abstract ideas, e.g., a bid or a bubble hierarchy, without some claimed 
practical application, Schrader, 22 F.3d at 293-94, 30 USPQ2d at 1458-59; 
Warmerdam, 33 F.3d at 1360, 31 USPQ2d at 1759. 

Mathematical algorithms have been held to be non-statutory because they 
merely describe an abstract idea. An "abstract idea" may simply be any sequence of 
mathematical operations that are combined to solve a mathematical problem. The 
concern addressed by holding such subject matter non-statutory is that the 
mathematical operations merely describe an idea and do not define a process that 
represents a practical application of the idea. 

Regarding claim 10, which recites "a matching algorithm comprising 
GetNextMatch( ), AssignNextMatch( ), and ShiftNextMatch( ) procedures, wherein: 
said GetNextMatch( ) procedure comprises the steps: 

testqueue: if queue. Empty( ); is a method to select a variable that 

is unassigned" in claim 10 is a computer program per se. Claim 10 is directed to 
a mere program listing, i.e., to only its description or expression, it is descriptive 
material per se and hence nonstatutory. (MPEP 2106 IV.B.1 (a)). 
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Claim Rejections - 35 USC § 103 



4. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, rf the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

This application currently names joint inventors. In considering patentability of 
the claims under 35 U.S.C. 103(a), the examiner presumes that the subject matter of 
the various claims was commonly owned at the time any inventions covered therein 
were made absent any evidence to the contrary. Applicant is advised of the obligation 
under 37 CFR 1 .56 to point out the inventor and invention dates of each claim that was 
not commonly owned at the time a later invention was made in order for the examiner to 
consider the applicability of 35 U.S.C. 103(c) and potential 35 U.S.C. 102(e), (f) or (g) 
prior art under 35 U.S.C. 103(a). 

5. Claims 1 and 11 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Non-Patent Document: "MediaNet: A multimedia Information Network for 
Knowledge Representation", In Conference on Internet Multimedia Management 
Systems. Vol. 4210 (numbered Pages 1-12), Boston, MA, Nov. 2000, 1 
ST/SPIE.00, Author: Benitez et al (hereinafter "Benitez") in view of Non-Patent 
Document: "Supporting Ranked Boolean Similarity Queries in MARS", IEEE 
Trans, on Knowledge and date Engineering, 10, Nov - Dec. 1998, Author: 
Ortaga et al (hereinafter "Ortega"). 
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Regarding claims 1 and 1 1 , Benitez teaches a method for querying stored 
multimedia data in a computer system, comprising: 

translating, in said intermediate level, said high-level concept into low-level queries 
by using system pre-defined high-level concepts (see section 3 "Typical content-based. 
In the database or are used in the retrieval" and section 4.2 page 8 "The extended 
content-based, the procedure is shown in Figure 1"); and 

transferring said low-level queries to a low level comprising one or more search 
engines; said one or more search engines performing a query of the stored multimedia 

information using said low-level queries (see section 4.2, page 9, "At this point and 

the results integrated into a unique list as described for visual queries"). 

Benitez does not explicitly teach receiving into an intermediate level a high-level 
concept from a user describing data to be retrieved. 

Ortega teaches receiving into an intermediate level a high-level concept from a user 
describing data to be retrieved (see section 1 .2, page 4, and paragraph 1 , "A Boolean 
... image databases"). 

It would have been obvious to one of ordinary skill in the art at the time the invention 
was made to combine teaching of Ortega with the teaching of Benitez wherein high 
level concepts can be expressed as a Boolean combination of lower level features. The 
motivation is that the user for the support of conceptual queries can provide such 
mapping of high to low level concepts explicitly. 
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6. Claims 2 - 9, and 12 - 19 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Benitez in view of Ortaga and further in view of Non-Patent 
Document: "CAMEL: Concept Annotated image Libraries", In Storage and 
Retrieval for Image and Video Database, San Jose, CA, Jan. 2001. SPIE 
(numbered pages 1-12), Author: Natsev et al (hereinafter "Natsev"). 

Regarding claims 2 and 12, Benitez and Ortega disclose the claimed subject 
matter as discussed in claims 1 and 1 1 respectively. Benitez or Ortega does not 
explicitly teach wherein said intermediate level comprises: a set of library modules, 
said set of library modules comprising: a concept library module for storing concepts; 

one or more library modules adapted to store said data from said one or more 
data sources; a cataloger module adapted to construct a new concept from said 
high-level concept using data from said concept library and library modules, thereby 
creating a concept construct, and to pass said concept construct to said concept 
library module for storage as a concept; and an interpreter module adapted to 
translate said high-level concept into low-level queries using said concepts stored in 
said construct library and to pass said low-level queries to said one or more search 
engines. 

Netsev teaches wherein said intermediate level comprises: a set of library 
modules, said set of library modules comprising: 

a concept library module for storing concepts (see abstract, page 1); 
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one or more library modules adapted to store said data from said one or more 
data sources (see figure 1 , section 1 .2 paragraph 3 f page 3 - page 4, paragraph 1 , 
"The concept Library is a module for persistent storage of concepts. ...terminology"); 

a cataloger module adapted to construct a new concept from said high-level 
concept using data from said concept library and library modules, thereby creating a 
concept construct, and to pass said concept construct to said concept library module 
for storage as a concept (see fig.3, section 1 .2, page 3); and 

an interpreter module adapted to translate said high-level concept into low-level 
queries using said concepts stored in said construct library and to pass said low- 
level queries to said one or more search engines (see section 1 .2 page 4 "The 
querying phase simply. ...keyword query"). - 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to combine teaching of Natsev with the teaching of Benitez and 
Ortega wherein the concept interpreter looks up the representation of the visual 
concept from the concept library and uses it to query the database for images 
containing the relevant concept. The motivation is that once a concept is defined, it 
can be used to search for relevant images without having annotated each and every 
one of them. 



Regarding claims 3 and 13, Benitez, Ortega and Natsev disclose the claimed subject 
matter as discussed in claims 2 and 12 respectively. Ortega teaches wherein said set of 
library modules further comprises at least one library module selected from the group 
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comprising: a matching algorithm library module adapted to store matching algorithms 
(see section 4.8, pages 20 and 21 ). 

Benitez or Ortega does not explicitly teach a feature library module adapted to store 
multimedia features; and a constraint library module adapted to store feature 
constraints. 

Natsev teaches a feature library module adapted to store multimedia features (see 
section 1 .2, paragraph 2, page 3); and 

a constraint library module adapted to store feature constraints (see section 5, 
pages 7 and 8). 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to combine teaching of Natsev with the teaching of Benitez and 
Ortega wherein the concept interpreter looks up the representation of the visual concept 
from the concept library and uses it to query the database for images containing the 
relevant concept. The motivation is that once a concept is defined, it can be used to 
search for relevant images without having annotated each and every one of them. 

Regarding claims 4 and 14, Natsev teaches wherein each said library module further 
comprises an application program interface to receive said data from a said data source 
(see section 4, page 7, "The last piece ... .in Figures 4 and 5."). 
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Regarding claims 5 and 15, Natsev teaches wherein said cataloger module further 
performs the steps of: 

selecting a set of concept features from said feature library module (see section 1 .2, 
paragraph 2, page 3); 

selecting a set of concepts from said concept library module for use as child- 
concepts (see section 6, paragraph 3, page 11); and 

selecting a set of constraints on said child concepts from said constraint library 
module (see section 6, paragraph 3, page 11). 

Regarding claims 6 and 16, Natsev teaches wherein said each said concept 
comprises a triplet of a set of child-concepts, a set of features, and a set of relationships 
(see section 6, paragraph 3, page 11, "Another improvement . . . ranked higher"). 

Regarding claims 7 and 17, Ortega teaches wherein said concepts comprise a 
hierarchical fuzzy graph data tree-structure comprising nodes, aggregation edges, and 
association edges and wherein: said nodes correspond to said concepts and said 
features; said aggregation edges correspond to parent-child relationships; and said 
association edges correspond to said constraints (see section 4.1, pages 10 and 11). 

Regarding claims 8 and 18, Ortega teaches wherein said edges are weighted (see 
section 4.2 page 11). 
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Regarding claims 9 and 19, Ortega teaches a matching algorithm comprising 
GetNextMatch( ), AssignNextMatch( ), and ShiftNextMatch( ) procedures, wherein: 
said GetNextMatch( ) procedure comprises the steps: 
testqueue: if queue. Empty( ); 

return NULL, 

head --> queue. Pop( ); 

if head.Complete( ); 

return head; head2 --> head.Copy( ); 

head2.AssignNextMatch( ); 
if head2.Valid( ); 

queue. Push(head2); 

head.ShiftNextMatch( ); 

queue. Push(head); 

Goto testqueue; 



said Assign NextMatch( ) procedure comprises the steps: 
child --> GetNextUnassigned( ); child.match_ptr++; 
if(child.match_ptr] == NULL), then; child. match_ptr --> 

child.GetNextMatch( ); 

Make child an assigned node; 



said ShiftNextMatch( ) procedure comprises the steps: 



Child -> GetNextUnassigned( ); 



child. match_ptr++; 
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if(child.match_ptr == NULL), then; 



child.match_ptr -> child. GetNextMatch( ); 



wherein variables head, head2, and child, all correspond to concept nodes; 
variable queue denotes a priority queue of the corresponding concept node; and 
match_ptr is a pointer to the next possible match for a given concept node; Pop( ) is a 
method to get the next node off the priority queue; Push( ) is a method to put a node on 
the priority queue; Empty( ) is a method to check if the priority queue is empty; Copy( ) 
is a method to copy a node; Complete( ) is a method to check if the children assignment 
is complete; Valid( ) is a method to check if the children assignment meets the 
constraints; and GetNextUnassigned( ) is a method to select a variable that is 
unassigned (see sections 4.6 - 4.9; pages 15 - 26). 

Nonfunctional descriptive material cannot render nonobvious an invention that 
would have otherwise been obvious. Cf. In re Gulack, 703 F.2d 1381, 1385, 217 
UPPQ 401, 404, (Fed. Cir. 1983) (When descriptive material is not functionally 
related to the substrate, the descriptive material will not distinguish the invention 
from the prior art in terms of patentability). The difference between the prior art and 
the claimed invention is simply a rearrangement of non-functional descriptive 
material (MPEP2106 VI). 

Therefore, It would have been obvious to one of ordinary skill in the art at the 
time the invention was made to rearrange the algorithm of Ortega to perform the 
function of GetNextMatch( ), AssignNextMatch( ), and ShiftNextMatch( ) procedures. 
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Conclusion 



7. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Fred I. Ehichioya whose telephone number is 703-305- 
8039. The examiner can normally be reached on M - F 8:00 AM to 4:30 PM. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Kim Y. Vu can be reached on 703-305-4393. The fax phone number for the 
organization where this application or proceeding is assigned is 703-746-7239. 

Any inquiry of a general nature or relating to the status of this application or 
proceeding should be directed to the receptionist whose telephone number is 703-303- 
3900. 

Fred I. Ehichioya 
Examiner 

Art Unit 2172 , 
December 1 1 , 2003 i / 




